Table narration using narration templates

ABSTRACT

A computer system for narrating a table using at least one narration template, wherein the table is extracted from a data source is provided. The computer system may include parsing the extracted table. The computer system may also include performing structural analysis on the parsed extracted table. The computer system may further include selecting at least one structural template based on the structural analysis of the parsed extracted table. Additionally, the computer system may include selecting the at least one narration template based on the at least one selected structural template. The computer system may also include applying the at least one selected narration template to the extracted table. The computer system may further include narrating the extracted table based on the applying of the at least one selected narration template to the extracted table.

FIELD OF THE INVENTION

The present invention relates generally to the field of computing, andmore particularly to table narration

BACKGROUND

Unstructured data sources and web pages may contain semi-structuredinformation in the form of tables. The tables may contain differenttypes of structured and unstructured data such as unformatted text,formatted text, photos or images, URLs or links, etc. Tables may benarrated so that the original table contents containing semi-structuredor structured data may be converted into a set of equivalent naturallanguage sentences and paragraphs. However, different tables may havevarious types of structural patterns and different types of contentwhich may require a different type of narration for each table.

SUMMARY

According to one embodiment, a method for narrating a table using atleast one narration template, wherein the table is extracted from a datasource is provided. The method may include parsing the extracted table.The method may also include performing structural analysis on the parsedextracted table. The method may further include selecting at least onestructural template based on the structural analysis of the parsedextracted table. Additionally, the method may include selecting the atleast one narration template based on the at least one selectedstructural template. The method may also include applying the at leastone selected narration template to the extracted table. The method mayfurther include narrating the extracted table based on the applying ofthe at least one selected narration template to the extracted table.

According to another embodiment, a computer system for narrating a tableusing at least one narration template, wherein the table is extractedfrom a data source is provided. The computer system may include one ormore processors, one or more computer-readable memories, one or morecomputer-readable tangible storage devices, and program instructionsstored on at least one of the one or more storage devices for executionby at least one of the one or more processors via at least one of theone or more memories, wherein the computer system is capable ofperforming a method. The method may include parsing the extracted table.The method may also include performing structural analysis on the parsedextracted table. The method may further include selecting at least onestructural template based on the structural analysis of the parsedextracted table. Additionally, the method may include selecting the atleast one narration template based on the at least one selectedstructural template. The method may also include applying the at leastone selected narration template to the extracted table. The method mayfurther include narrating the extracted table based on the applying ofthe at least one selected narration template to the extracted table.

According to yet another embodiment, a computer program product fornarrating a table using at least one narration template, wherein thetable is extracted from a data source is provided. The computer programproduct may include one or more computer-readable storage devices andprogram instructions stored on at least one of the one or more tangiblestorage devices, the program instructions executable by a processor.

The computer program product may also include program instructions toparse the extracted table. The computer program product may also includeprogram instructions to perform structural analysis on the parsedextracted table. The computer program product may further includeprogram instructions to select at least one structural template based onthe structural analysis of the parsed extracted table. Additionally, thecomputer program product may include program instructions to select theat least one narration template based on the at least one selectedstructural template. The computer program product may also includeprogram instructions to apply the at least one selected narrationtemplate to the extracted table. The computer program product mayfurther include program instructions to narrate the extracted tablebased on the applying of the at least one selected narration template tothe extracted table.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

These and other objects, features and advantages of the presentinvention will become apparent from the following detailed descriptionof illustrative embodiments thereof, which is to be read in connectionwith the accompanying drawings. The various features of the drawings arenot to scale as the illustrations are for clarity in facilitating oneskilled in the art in understanding the invention in conjunction withthe detailed description. In the drawings:

FIG. 1 illustrates a networked computer environment according to oneembodiment;

FIG. 2 illustrates the basic elements or constructs of a table accordingto one embodiment;

FIG. 3 illustrates an example of horizontal narration according to oneembodiment;

FIG. 4 illustrates an example of horizontal narration with comparisonaccording to one embodiment;

FIG. 5 illustrates an example of vertical narration according to oneembodiment;

FIG. 6 illustrates an example of vertical narration with comparisonaccording to one embodiment;

FIG. 7 illustrates a template-based approach for linking structuralanalysis and narration for tables according to one embodiment;

FIGS. 8A-8B is an operational flowchart illustrating the steps carriedout by a program for narrating a table using narration templatesaccording to at least one embodiment;

FIG. 9 illustrates examples of narration formulas according to oneembodiment;

FIG. 10 is a block diagram of internal and external components ofcomputers and servers depicted in FIG. 1 according to at least oneembodiment.

DETAILED DESCRIPTION

Detailed embodiments of the claimed structures and methods are disclosedherein; however, it can be understood that the disclosed embodiments aremerely illustrative of the claimed structures and methods that may beembodied in various forms. This invention may, however, be embodied inmany different forms and should not be construed as limited to theexemplary embodiments set forth herein. Rather, these exemplaryembodiments are provided so that this disclosure will be thorough andcomplete and will fully convey the scope of this invention to thoseskilled in the art. In the description, details of well-known featuresand techniques may be omitted to avoid unnecessarily obscuring thepresented embodiments.

Embodiments of the present invention relate generally to the field ofcomputing, and more particularly to table narration. The followingdescribed exemplary embodiments provide a system, method and programproduct for table narration using narration templates.

As previously described, unstructured data sources, such as documentsand web pages (e.g., HTML or XML), may contain semi-structuredinformation in the form of tables. The tables may contain differenttypes of structured and unstructured data such as unformatted text,formatted text (e.g., paragraphs, sentences, bulleted or numberedlists), photos or images, URLs or links, etc. Currently, tables may benarrated so that the original table contents containing semi-structuredor structured data may be converted into a set of equivalent naturallanguage sentences and paragraphs. However, different tables may havevarious types of structural patterns and different types of content. Assuch, it may be very difficult to narrate all types of tables in ageneralized manner. Furthermore, there currently is no generic approachavailable which may be used to narrate all types of tables withdifferent structural patterns and different types of content in astandardized and consistent manner. Also, current table narrationapproaches may not be flexible and scalable, and may not be based ontemplates. As such, the current table narration approaches may requirecustom programming based in different table structures and contentssince the current table narration approaches may not be based ontemplates. Therefore, it may be advantageous, among other things, toprovide a flexible and scalable solution which may minimize code changesand may generalize table narration for all types of tables and tablestructures.

Table narration may be the conversion and description of the contents ofa table (i.e., semi-structured data) into free form natural languagesentences and paragraphs, so that the resulting narration may beequivalent to the original table contents and meaning. According to atleast one embodiment, table narration may be implemented using naturallanguage processing (NLP) concepts. The present embodiment may definethe concept of narration templates and the key elements (i.e., schema)for narration templates and structural templates. The present embodimentmay also define the concept of a narration formula, and its usage tonarrate a table at a cell level, row level, column level or any othertype of generic region defined within the table. As such, the narrationformula may make use of NLP (natural language processing concepts) andthus NLP may be used to narrate tables in a generic way. The narrationformula may also be modified as required to improve the narration,depending on the table contents.

Furthermore, according to at least one embodiment, the linkage of anarration template to the structure of a table through the definitionand use of structural templates and a mapping mechanism may beimplemented. Also, structural templates and narration templates may bestored in a repository (such as a database or a file system), and eachstructural template may map to, be linked with, or be associated withone or more narration templates through a “template mapping”, such as amapping file, hashmap, dictionary, linked list or any other type ofmapping mechanism which may be used to store the mapping between astructural template and a narration template or the mapping between anarration template and a structural template. The present embodimentsmay define the concepts of horizontal narration; vertical narration;horizontal narration with comparison of a narration template across rowsor groups of rows; and vertical narration with comparison across columnsor groups of columns in a table. Additionally, the present embodimentsmay define how the narration templates may be used for these varioustypes of narration in a generic way.

According to at least one implementation, the narration templates may belinked to the structure of a table by mapping structural templates andnarration templates. As such, the mapping of the structural templatesand the narration templates may help in determining the narrationtemplates applicable and relevant to a particular table based on itsstructural pattern. Furthermore, tables in documents or web pages may beextracted, parsed and analyzed for their structure and layout. Thenbased on the structural analysis of a table, the applicable “structuraltemplate” may be determined. The “structural template” may depict thekey elements of the table structure that are necessary to determine theapplicable narration templates. Therefore, based on the structuraltemplate determined for the given table, the set of (one or more)applicable narration templates may be determined by looking up the“template mapping”. Furthermore, for a given structural template, ifmore than one narration template may be applicable, then additionalrules or criteria may be used to deselect or select one narrationtemplate from the set of applicable narration templates. Additionally,the given table may be narrated by “applying” the selected narrationtemplate to the table to get the resultant table narration.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The following described exemplary embodiments provide a system, methodand program product for table narration using narration templates. Aspreviously described, table narration is the conversion and descriptionof the contents of a table (i.e., semi-structured data) into free formnatural language sentences and paragraphs, so that the resultingnarration may be equivalent to the original table contents and meaning.According to the present embodiment, narration templates, structuraltemplates, and the corresponding schemas may be implemented.Additionally, a narration formula using natural language processing(NLP) techniques may be implemented. As such, the narration formulas maybe used to narrate a table in a generic way at cell level, row level,column level or any other type of generic region defined within thetable. Therefore, the generic mechanism may link and map narrationtemplates to structural templates. Additionally, according to at leastone implementation, narration techniques, such as horizontal narration;vertical narration; horizontal narration with comparison across rows orgroups of rows; and vertical narration with comparison across columns orgroups of columns in a table may be implemented and used for thesevarious types of narration in a generic way.

Referring now to FIG. 1, an exemplary networked computer environment 100in accordance with one embodiment is depicted. The networked computerenvironment 100 may include a computer 102 with a processor 104 and adata storage device 106 that is enabled to run a table narration program108A. The networked computer environment 100 may also include a server112 that is enabled to run a table narration program 108B and acommunication network 110. The networked computer environment 100 mayinclude a plurality of computers 102 and servers 112, only one of whichis shown for illustrative brevity. The communication network may includevarious types of communication networks, such as a wide area network(WAN), local area network (LAN), a telecommunication network, a wirelessnetwork, a public switched network and/or a satellite network. It may beappreciated that FIG. 1 provides only an illustration of oneimplementation and does not imply any limitations with regard to theenvironments in which different embodiments may be implemented. Manymodifications to the depicted environments may be made based on designand implementation requirements.

The client computer 102 may communicate with server computer 112 via thecommunications network 110. The communications network 110 may includeconnections, such as wire, wireless communication links, or fiber opticcables. As will be discussed with reference to FIG. 10, server computer112 may include internal components 800 a and external components 900 a,respectively and client computer 102 may include internal components 800b and external components 900 b, respectively. Client computer 102 maybe, for example, a mobile device, a telephone, a personal digitalassistant, a netbook, a laptop computer, a tablet computer, a desktopcomputer, or any type of computing device capable of running a programand accessing a network.

A program, such as a table narration program 108A and 108B may run onthe client computer 102 or on the server computer 112. The tablenarration program 108A, 108B may be utilized to narrate a table usingnarration templates. For example, a user using a table narration program108A, running on a client computer 102 may connect via a communicationnetwork 110 to server computer 112 which may also be running a tablenarration program 108B. Furthermore, the user using client computer 102or server 112 may utilize the table narration program 108A, 108B togenerically narrate a table using narration templates. The tablenarration method is explained in further detail below with respect toFIGS. 7A-7B.

Referring now to FIG. 2, an example of the basic elements or constructsof a table 200 in accordance with one embodiment is depicted. Aspreviously explained, table processing and narration in a generic mannermay be a complex problem, and currently there are no known solutions fornarration of all different types of tables in a generic way, withoutcustom coding. Tables may be represented in multiple formats, dependingon the source document formats such as HTML, PDF, documents,spreadsheets, presentations, XML, etc. Additionally, tables may havevarious types of structural patterns (e.g., combination of cells, rowsand columns) in different layouts (e.g., tables with a caption 202,tables with normal cells 204 tables with one header row 206, tables withone header column 208, tables with one header row 206 and one headercolumn 208, tables with a spanned header cell 210, tables with asub-header row 212, tables with a spanned cell 214, tables with multiplelevels of sub-header rows 212, tables with intermediate summary rows orfooter rows 216, tables with multiple levels of category andsub-category rows 218, tables with random headings, etc.) Furthermore,tables may contain semi-structured data with various content types suchas words, phrases, sentences, lists (e.g., bulleted or numbered),paragraphs, numbers or financials, rich content (e.g., images, graphs,charts), etc.

Referring now to FIG. 3, an example of horizontal narration 300 inaccordance with one embodiment is depicted. According to at least oneimplementation of the present embodiment, horizontal narration may beused to narrate rows of a table (right to left or left to right)including linking the contents in header rows 302 through narrationformulas using NLP. Therefore, if rows of a table are to be narrated(left to right for English language, or right to left for certainlanguages such as Arabic), it is referred to as horizontal narration. Atable may have key elements, such as a caption 304, a header row 302, aheader column 306, etc. According to at least one implementation, eachnormal row of a table 308 may be converted to one or more equivalentnatural language sentences. Therefore, according to at least oneimplementation, the cell values in normal rows 308 of a table may belinked to the headers and sub-headers (e.g., header rows 302; headercolumns 306; category rows 218 (FIG. 2); intermediate summary rows; andfooter rows 216 (FIG. 2)) of a table using natural language processing(NLP) concepts using narration formulas.

For example, with respect to FIG. 3, the output following horizontalnarration of the marked header rows 302 may be displayed as thefollowing:

For Mutual Fund Fund 1, the Dividend ($ per unit) is $2 and Payout % is2%.

For Mutual Fund Fund 2, the Dividend ($ per unit) is $4 and Payout % is1.8%.

For Mutual Fund Fund 3, the Dividend ($ per unit) is $6 and Payout % is8%.

For Mutual Fund Fund 4, the Dividend ($ per unit) is $20 and Payout % is15%.

Referring now to FIG. 4, an example of horizontal narration withcomparison 400 in accordance with one embodiment is depicted. Accordingto at least one implementation of the present embodiment, horizontalnarration with comparison may be used to compare one row or one group ofrows of a table (right to left or left to right) with another row (oranother group of rows) including linking the contents in header rows 302through narration formulas using NLP. Therefore, if a table containsmultiple rows (or repeating groups of rows), and the horizontalnarration for one row (or one group of rows) should be compared with thehorizontal narration for another row (or a second group of rows), it isdefined as “Horizontal Narration Comparison”. However, according to analternate implementation, the horizontal narration may compare one rowof a table with another row (or a set of rows) of the table. As such,the cell values in normal rows of a table may be linked to the headersand sub-headers (e.g., header row 402, row 3 404, and row 5 406); headercolumns 306; category rows 218 (FIG. 2); intermediate summary rows; andfooter rows 216 (FIG. 2)) of a table using natural language processing(NLP) concepts using narration formulas according to the presentembodiment.

For example, with respect to FIG. 4, the output following horizontalnarration with comparison across row 3 (404) and row 5 (406) may bedisplayed as the following:

For Mutual Fund Fund 2, the Dividend ($ per unit) is $4 and Payout % is1.8%.

Whereas

For Mutual Fund Fund 4, the Dividend ($ per unit) is $20 and Payout % is15%.

Referring now to FIG. 5, an example of vertical narration 500 inaccordance with one embodiment is depicted. According to at least oneimplementation of the present embodiment, vertical narration may be usedto narrate columns 502 of a table (top-down or bottom-up) includinglinking the contents in header rows (302 (FIG. 3)) through narrationformulas using NLP. Therefore, if columns 502 of a table are to benarrated in a top-down or bottom-up sequence, it may be referred to asvertical narration. As such, each column 502 of a table may be convertedto one or more equivalent natural language sentences. According to atleast one implementation, the cell values in normal columns 502 of atable may be linked to the headers and sub-headers (header row 402 (FIG.4); header columns; 306 (FIG. 3) category rows (218 (FIG. 2);intermediate summary rows; and footer rows (216 (FIG. 2)) of a tableusing natural language processing (NLP) concepts.

For example, with respect to FIG. 5, the output following verticalnarration of the marked columns 502 may be displayed as the following:

Definition for Four Wheeler is Any Vehicle that has 4 wheels such ascar, bus, truck, etc.

Policy Type for Four Wheeler is a Four Wheeler Insurance.

Referring now to FIG. 6, an example of vertical narration withcomparison 600 in accordance with one embodiment is depicted. Accordingto at least one implementation of the present embodiment verticalnarration may be used to narrate columns 502 of a table (top-down orbottom-up) including linking the contents in header rows 602 throughnarration formulas using NLP. Therefore, if a table contains multiplecolumns 502 (or repeating groups of columns) which can be verticallynarrated, and the vertical narration for one column 502 (or one group ofcolumns) should be compared with the vertical narration for anothercolumn (or a second group of columns), it may be defined as “VerticalNarration Comparison”. According to another implementation of thepresent embodiment, the vertical narration may compare one column 608 ofa table with another column 610 (or a set of columns) of the table. Assuch, the cell values in normal cells of column 502 of a table may belinked to the headers and sub-headers (header row 302 (FIG. 3)); headercolumns 306 (FIG. 3); category rows 218 (FIG. 2); intermediate summaryrows; and footer rows 216 (FIG. 2)) of a table using natural languageprocessing (NLP) concepts according to the present embodiment.

For example, with respect to FIG. 6, the output following verticalnarration with comparison across the columns 604 (column 2) and 606(column 3) of a table, may be displayed as the following:

Definition:

Four Wheeler is any vehicle that has 4 wheels such as car, bus, truck,etc.

Whereas

Two Wheeler is any vehicle that has 2 wheels such as bicycle, motorbike,etc.

Policy Type for Four Wheeler is a Four Wheeler Insurance.

Whereas

Policy Type for Two Wheeler is a Bike Insurance.

According to an alternate embodiment, narration formulas for narratingtables corresponding to specified structural and narration templates maybe implemented. When a structural template and the applicable narrationtemplate are determined for a given table, the narration formulaspecified in the selected narration template may be “applied” to narratethe given table. As such, a narration formula may consist of a mix ofnatural language words, phrases, and sentences which can be static(specified within the formula) OR dynamic (based on the contents of agiven table) through NLP techniques (such as determination of hypernymsor super-subordinate relationships among cognitive synonyms, prepositionclasses, plurality, etc.) or based on values specified in otherparameters of the narration template. Additionally, variables may beresolved to header cells, normal cells, rows 302 (FIG. 3), columns 502(FIG. 5) or other structural elements of a table, such that the contentsof the element specified by the given variable may be accessed andsubstituted within the formula.

Additionally, according to the present embodiment, narration formulasmay be written in any formula language, code or notation that may beinterpreted and resolved at runtime by the narration program. The choiceof formula language or notation may be implementation-specific. Examplesof narration formulas and their explanations are provided in FIG. 9.These are sample embodiments, and other types of narration formulas mayalso be defined, with different syntax or notations.

Referring now to FIG. 7, a template-based approach 700 for linkingstructural analysis and narration for tables in accordance with oneembodiment is depicted. The method may include data sources for tables702; extraction and structural analysis of a table 704; a tablestructure object 706; and mapping 708. The method may further include astructural template 710; a narration template 712; a table narrationobject 714; a table narration output 716; an ingestion pipeline 718; anda question answer system 720. The method may further include the storingof templates in repositories, such as a structural templates repository722; a template mapping repository 724; and a narration templatesrepository 726.

For example, according to the present embodiment, at 702, a table may belocated in source documents or web pages. Then at 704, extraction andstructural analysis of the table may be performed. As such, the tablesin the source documents or web pages may be extracted, parsed andanalyzed for their structure and layout.

Then at 706, the table structure object may be determined. As such,based on the previous structural analysis of the table and using thestructural templates repository 722, the applicable “structuraltemplate” may be determined. The “structural template” may depict thekey elements (i.e., schema) of the table structure that may be necessaryto determine the applicable narration templates. Then, at 708, based onthe structural template determined for the given table, the set of (oneor more) applicable narration templates may be determined by looking upthe “template mapping” in the template mapping repository 724.Therefore, at 710, each structural template may be mapped to one or morenarration templates and at 712 each narration template may be mapped toone or more structural templates.

Furthermore, for a given structural template, if more than one narrationtemplate may be applicable, then additional rules or criteria may beused to downselect or select one narration template from the set ofapplicable narration templates 726. The given table may be narrated bythe method of “applying” the selected narration template to the table at714 to get the resultant table narration at 716. As such, the presentembodiment may provide flexibility for table narration in an ingestionpipeline 718 for use in a question answering system 720.

For example, according to the present embodiment, schema, templates andmapping of narration templates may be implemented as follows:

Schema

-   -   Narration Schema: This may contain the key elements (including        NLP elements) for defining narration templates, including a        narration formula (according to one implementation) which may be        based on NLP concepts.    -   Structural Schema: This may contain the key elements of a table        structure which may be required for determining the relevant        narration templates that may be applicable for a given table.        The structural schema may or may not represent the entire table        structure, since the entire table structure may not be required        to determine the mapping between a structural template and a        narration template.    -   Mapping Schema: This may contain the key elements required to        map a narration template to a structural template (and vice        versa). The mapping schema may also contain certain elements        such as domain name or client name which may be used to        differentiate a set of narration templates from other narration        templates for the same structural table schema.

Template

-   -   Narration Template: This may comply with the Narration Schema,        and defines the narration elements (including NLP elements)        which are used to derive the narration of a given table.    -   Structural Template: This may comply with the Structural Schema,        and defines the structural elements of a table which are        relevant for deriving the corresponding table narration.    -   Mapping Template: This may comply with the Mapping Schema, and        is used to map the structural templates to the narration        templates (and vice versa).

According to one embodiment, an example of one implementation depictingthe structural schema may be as follows:

Element Type/ Schema Element Description Data Type Table Structure Thisis a complex element containing the structural Complex Template templatename and template version. Element Table Structure This is the name ofthe structural template; it may be the String Template Name same as thefilename (with or without the file extension), or any other name whichuniquely identifies the template. Table Structure The template versionidentifies the version of the String Template Version template, and canbe used for version control. Table Structural This is a complex elementcontaining all the child Complex type Pattern elements for thestructural pattern representing a table. Only those structural elementsof a table which are required to identify the corresponding narrationtemplates are included in this structural pattern. Table Structural Thisis the name of the structural pattern, and should be a String PatternName unique name. Table Header Row This indicates whether the tablecontains a header row, String (Yes/No) Exists and may be a Boolean(Yes/No) type of value. Table Header Row This indicates whether thetable contains sub-header String (Yes/No) SubHeader Exists rows, and maybe a Boolean (Yes/No) type of value. Table Header Row If the tablecontains sub-header rows, then this indicates Integer SubHeader Levelsthe number of sub-levels. If the table does not contain sub-header rows,the value would be zero. Table Header Row In many cases, the cell at theintersection of the sub- String (Yes/ SubHeader First header rows andthe first column of the table is blank. No/NA) Cell Blank This elementindicates if the cell is blank or not, and may be a Boolean (Yes/No)type of value. Table Header This indicates whether the table contains aheader String (Yes/No) Column Exists column, and may be a Boolean(Yes/No) type of value. Table Header This indicates whether the tablecontains sub-header String (Yes/No) Column columns, and may be a Boolean(Yes/No) type of value. SubHeader Exists Table Header If the tablecontains sub-header columns, then this Integer Column indicates thenumber of sub-levels. If the table does not SubHeader Levels containsub-header columns, the value would be zero. Table Header In many cases,the cell at the intersection of the sub- String (Yes/ Column headercolumns and the first row of the table is blank. No/NA) SubHeader FirstThis element indicates if the cell is blank or not, and may Cell Blankbe a Boolean (Yes/No) type of value. Table Header Row This indicates themaximum column span for a header Integer Maximum Column row. The valuewould be 1 if the table does not contain Span any spanned columns. TableHeader This indicates the maximum row span for a header Integer ColumnMaximum column. The value would be 1 if the table does not Row Spancontain any spanned rows. Table Header Row This indicates whether thetable contains a header row String (Yes/No) Group Exists and groups ofrepeating columns, and may be a Boolean (Yes/No) type of value. TableHeader Row If the table contains groups of repeating columns, thenInteger Group Start this indicates the column number where the repeatingColumn group starts (the first column of the first repeating group). Avalue of 0 indicates that there is no repeating group of columns in thetable. Example - (a) if the table contains 4 columns with a repeatinggroup of 2 columns (A B A B), then the starting column number is 1. (b)If the repeating pattern is (C1 A B A B) where the first column C1 isnon-repeating, then the start column number for the repeating pattern (AB) is 2. Table Header Row If the table contains groups of repeatingcolumns, then Integer Group Size the size of a group (i.e. the number ofcolumns in the repeating group). A value of 0 indicates that there is norepeating group of columns in the table. Example - if the table contains4 columns with a repeating group of 2 columns (A B A B), then the groupsize is 2. Table Header This indicates whether the table contains aheader String (Yes/No) Column Group column and groups of repeating rows,and may be a Exists Boolean (Yes/No) type of value. Table Header If thetable contains groups of repeating rows, then this Integer Column Groupindicates the row number where the repeating group Start Row starts (thefirst row of the first repeating group). A value of 0 indicates thatthere is no repeating group of rows in the table. Example - (a) if thetable contains 4 rows with a repeating group of 2 rows (X Y X Y), thenthe starting row number is 1. (b) If the repeating pattern is (H1 X Y XY) where the first row H1 is a header row and non- repeating, then thestart row number for the repeating pattern (X Y) is 2. Table Header Ifthe table contains groups of repeating rows, then the Integer ColumnGroup size of a group (i.e. the number of rows in the repeating Sizegroup). A value of 0 indicates that there is no repeating group of rowsin the table. Example - if the table contains 4 rows with a repeatinggroup of 2 rows (X Y X Y), then the group size is 2. Table Category Thisindicates whether the table contains category rows, String (Yes/No)Headers Exist and may be a Boolean (Yes/No) type of value. TableCategory If the table contains category rows, this indicates if it alsoString (Yes/No) SubLevels Exist has category sub-levels. This may be aBoolean (Yes/ No) type of value. Table Category If the table hascategory sub-levels, this indicates the Integer Maximum maximum numberof sub-levels. A value of 0 indicates SubLevels that the table does notcontain category sub-levels. Table Intermediate This indicates whetherthe table contain intermediate String (Yes/No) Summary Rows summaryrows, and may be a Boolean (Yes/No) type of Exist value. (Intermediatesummary rows are rows which contain summaries or totals of a set of rowsimmediately preceding or succeeding them. A common example is invoicesor balance sheets where a set of items may have an intermediate totalrow.) Table Footer This indicates whether the table contains a footerString (Yes/No) Summary Row summary row, and may be a Boolean (Yes/No)type of Exists value. (A footer row is the last row of a table whichcontains summaries or totals of a set of rows preceding it in the table.A common example is invoices or balance sheets where a set of items mayhave a total in the last row.) Table Random This indicates whether thetable contains randomly String (Yes/No) Headings Exist placed headings,and may be a Boolean (Yes/No) type of value. (Normal tables may containa header row as the first row, and each cell of a header row acts as theheader cell for the corresponding column. However, there may be tablepatterns where a cell acts as the heading for a group of cells below it,and such heading cells are not necessarily the first cell in a column.The placement and occurrence of such heading cells may be random.)

According to another embodiment, an example of one implementationdepicting the narration schema may be as follows:

Element Type/ Schema Element Description Data Type Table Narration Thisis a complex element containing the narration Complex Template templatename and template version. Element Table Narration This is the name ofthe narration template; it may be the String Template Name same as thefilename (with or without the file extension), or any other name whichuniquely identifies the template. Table Narration The template versionidentifies the version of the String Template Version template, and canbe used for version control. Table Narration This is a complex elementcontaining all the child Complex Pattern elements for the narrationpattern for a table. Element Table Narration This is the name of thenarration pattern, and should be a String Pattern Name unique name.Table Narration For tables containing a header row, this element definesString Header Row Type the type of text contained within the cells ofthe header row. The text could be a noun phrase, or it could start witha preposition, or it could be numeric. In case a table does not containa header row, this element can have the value of 'Not Applicable'. Othervalues can be defined as required. Table Narration For tables containingsub-header rows, this element String Header Row defines the type of textcontained within the cells of the SubHeader Type sub-header rows. Thetext could be a noun phrase, or it could start with a preposition, or itcould be numeric. In case a table does not contain a sub-header row,this element can have the value of 'Not Applicable'. Other values can bedefined as required. Table Narration For tables containing a headercolumn, this element String Header Column defines the type of textcontained within the cells of the Type header column. The text could bea noun phrase, or it could start with a preposition, or it could benumeric. In case a table does not contain a header column, this elementcan have the value of ‘Not Applicable’. Other values can be defined asrequired. Table Narration For tables containing sub-header columns, thiselement String Header Column defines the type of text contained withinthe cells of the SubHeader Type sub-header columns. The text could be anoun phrase, or it could start with a preposition, or it could benumeric. In case a table does not contain a sub-header column, thiselement can have the value of ‘Not Applicable’. Other values can bedefined as required. Table Narration The header class is the hypernym(parent class or String Header Class category) that the text in thefirst cell of the header row belongs to. For example, if the first cellof a header row contains the text “Age”, then the header class could bean “attribute”. Other values of header classes may be defined asrequired, for example “set”, “measure”, “communication”, etc. If theheader class is not applicable, its value can be set to ‘NotApplicable’. (REFERENCE: Wordnet [http://wordnet.princeton.edu/wordnet/]uses hypernyms and provides super-subordinate relations (also calledhyperonymy, hyponymy or ISA relation) among cognitive synonyms orsynsets.) Table Narration This element indicates the plurality of thetext contained String Header Row within the cells of the header row. Thetext phrase or Plurality sentence may be singular or plural. For numericor other types of non-textual content, the value may be “NA” (NotApplicable). Other types of values may be defined if required. TableNarration This element indicates the plurality of the text containedString Header Column within the cells of the header column. The textphrase or Plurality sentence may be singular or plural. For numeric orother types of non-textual content, the value may be “NA” (NotApplicable). Other types of values may be defined if required. TableNarration This element specifies the classes into which prepositionsString Preposition Class can be categorized. Values may be defined asrequired. (REFERENCE: Preposition Classes are defined by The PrepositionProject - http://www.clres.com/prepositions.html) Table Narration Fortables containing category rows, this element defines String CategoryHeader the type of text contained within the cells of the category Typerows. The text could be a noun phrase, or it could start with apreposition, or it could be numeric. In case a table does not contain acategory row, this element can have the value of ‘Not Applicable’. Othervalues can be defined as required. Table Narration For tables containingcategory row sub-levels, this String Category SubLevel element definesthe type of text contained within the cells Type of the categorysub-levels. The text could be a noun phrase, or it could start with apreposition, or it could be numeric. In case a table does not contain acategory sub- levels, this element can have the value of ‘NotApplicable. Other values can be defined as required. Table NarrationThis element indicates the plurality of the text contained StringCategory Header within the cells of the category rows. The text phraseor Plurality sentence may be singular or plural. For numeric or othertypes of non-textual content, the value may be ‘Not Applicable’. Othertypes of values may be defined if required. Table Narration This elementindicates the narration sequence for a table. String Sequence Possiblevalues are: Horizontal: If rows of a table are to be narrated (left toright for English language, or right to left for certain languages suchas Arabic), it is referred to as horizontal narration. Vertical: Ifcolumns of a table are to be narrated in a top-down sequence, it isreferred to as vertical narration. Horizontal-Compare: If a tablecontains repeating groups of rows, and the horizontal narration for onegroup of rows should be compared with the horizontal narration foranother group of rows, it is referred to as “Horizontal ComparisonNarration”. Another variation of this could be that the horizontalnarration compares one row of a table with another row (or a set ofrows) of the table. Vertical-Compare: If a table contains repeatinggroups of columns, and the vertical narration for one group of columnsshould be compared with the vertical narration for another group ofcolumns, it is referred to as “Vertical Comparison Narration”. Anothervariation of this could be that the vertical narration compares onecolumn of a table with another column (or a set of columns) of thetable. Table Maximum This indicates the maximum number of rows in atable. Integer Rows For some types of narration templates (such asnarration of tables with a single row without header rows), this is animportant parameter to define the narration formula. A default value of0 or −1 (or any non-positive integer) for this parameter would indicatethat this parameter is not applicable. A positive integer value (1, 2,100, etc) would indicate the maximum number of rows for which thenarration template would be applicable. Table Maximum This indicates themaximum number of columns in a Integer Columns table. For some types ofnarration templates (such as narration of tables with a single column),this is an important parameter to define the narration formula. Adefault value of 0 or −1 (or any non-positive integer) for thisparameter would indicate that this parameter is not applicable. Apositive integer value (1, 2, 100, etc) would indicate the maximumnumber of columns for which the narration template would be applicable.Table Narration For tables containing footer rows, this element definesString Footer Summary the type of text contained within the cells of thefooter Type rows. The text could be a noun phrase or sentence, or itcould start with a preposition, or it could be numeric. In case a tabledoes not contain a footer row, this element can have the value of ‘NotApplicable’. Other values can be defined as required. Table NarrationThe footer summary class is the hypernym (parent class String FooterSummary or category) that the text in the cells of the footer row Classbelongs to. For example, if the footer row contains the summation ofvalues in the preceding rows of the table, then the footer summary classcould be “quantitative relation”. Other values of footer summary classesmay be defined as required, for example “set”, “measure”,“communication”, etc. If the footer summary class is not applicable, itsvalue can be set to ‘Not Applicable’. (REFERENCE: Wordnet[http://wordnet.princeton.edu/wordnet/] uses hypernyms and providessuper-subordinate relations (also called hyperonymy, hyponymy or ISArelation) among cognitive synonyms or synsets.) Table Narration Thiselement defines the type of text contained within String Cell Value Typeeach normal cell of a table. This is an essential parameter, sinceduring narration using our proposed approach, the contents of each cellof a table are correlated with the header rows/header columns/footerrows, etc using natural language processing (NLP) concepts, indetermining the cell level narration or row level narration. The textwithin a cell could be a noun phrase, a sentence, a list (such as abulleted list or a numbered list), the text could start with apreposition or the text could be numeric. In case the table cell doesnot contain a value (is blank), this element can have the value of ‘NotApplicable’. Other values can be defined as required. Table Cell LevelThis element represents a cell-level narration formula. String NarrationFormula Such formulas can be applied to individual table cells tonarrate the cell contents. Default value is “NIL” if the formula is notdefined at the cell level. Some types of narration templates may nothave cell-level narrations. If the formula is defined, then this elementwill contain the formula as a text string. Table Row Level This elementrepresents a row-level narration formula. String Narration Formula Suchformulas can be applied to table rows to narrate the row contents. A rowlevel narration formula may be applied to each normal row of a table(excluding header rows or category rows or footer rows) to produce arow-level narration. The narration may be a sentence. Thus each normalrow of a table may be narrated into (one or more) sentences using therow level narration formula. Default value is “NIL” if the formula isnot defined at the row level. Some types of narration templates may notneed row-level narrations. If the formula is defined, then this elementwill contain the formula as a text string. Table Footer Level Thiselement represents a row-level narration formula for String NarrationFormula footer rows or intermediate summary rows. Such formulas may beapplied to footer rows or intermediate summary rows to narrate the rowcontents. Since footer rows or intermediate summary rows are specialtypes of rows in a table, the footer level narration formula defined bythis element may not be used for other normal rows of a table. Defaultvalue is “NIL” if the formula is not defined. Most types of tables donot contain footers, and the corresponding narration templates may nothave footer- level narrations. If the formula is defined, then thiselement will contain the formula as a text string.

According to yet another embodiment, an example of one implementationdepicting the mapping schema may be as follows:

Minimum/ Data Maximum Schema Element Description Type Occurrences MapTable This is a complex element containing the Complex 1 StructureNarration mapping template name and template version. Element Map NameMapping template name; same as the String 1 filename (without fileextension.) Map Version Mapping Template version. String 1 TableStructure This is a complex element containing the Complex Min = 1Narration Mapping mapping between structural templates and Element Max =ANY narration templates. Each mapping file can contain one or multiplemapping elements. One mapping element can be used to map one structuraltemplate to one or multiple narration templates. If the mapping isspecific to a domain or to a specific company/customer/client, theseparameters can also be specified as part of this element. TableStructure The name of a structural template. This name String 1 TemplateName should match the name specified within the structural templatefile. A variation of this implementation may have the filename as thename of the structural template. Each mapping element (“Table StructureNarration Mapping”) can have exactly one structural template name. Thus,each structural template can be mapped to one or more narrationtemplates. Table Narration The name of a narration template. This nameString Min = 1 Template Name should match the name specified within theMax = ANY narration template file. A variation of this implementationmay have the filename as the name of the narration template. Eachmapping element (“Table Structure Narration Mapping”) can have one ormultiple narration template names. Mapping Domain This element specifiesa domain name for the String Min = 1 Name mapping element (“TableStructure Narration Max = ANY Mapping”), for example, “generic”,“telecom”, “healthcare”, etc. The default value is “generic”, whichmeans that the mapping of structural and narration templates is validfor all domains/is not restricted to any specific domain. Each mappingelement (“Table Structure Narration Mapping”) may have one or moredomain names associated with it. This enables a structural template andits associated mapping templates to be associated with the specifieddomains. Mapping Client This element specifies a company/customer/String Min = 1 Name client name for the mapping element (“Table Max =ANY Structure Narration Mapping”), for example, “all”, “IBM”, etc. Thedefault value is “all”, which means that the mapping of structural andnarration templates is valid for all companies/customers and is notrestricted to any specific company. Each mapping element (“TableStructure Narration Mapping”) may have one or more company or clientnames associated with it. This enables a structural template and itsassociated mapping templates to be associated with the specifiedcompanies or clients.

The foregoing schema definitions may be considered as a sampleimplementation of the present embodiment. As such, the foregoing schemadefinitions may vary for other implementations such that new elements orparameters may be added to the proposed schemas; certain elements orparameters discussed herein may be customized or modified; and certainelements or parameters discussed herein may be excluded or madeoptional. Therefore, different schemas may be defined for specificdomains, for specific clients, or for specific criteria. According to atleast one implementation, XSD (XML Schema Definition) may be utilized todefine schemas. Also, XML (Extended Markup Language) may be utilized todefine the structural and narration templates and the mapping file.Furthermore, each XML template file can comply with the correspondingXSD schema and multiple templates (XML) can be defined for the same XSDSchema.

Referring now to FIGS. 8A-8B, an operational flowchart 8000 illustratingthe steps carried out by a program for narrating a table using narrationtemplates in accordance one embodiment is depicted. At 8002, the inputtable to be narrated may be detected and extracted from a data source,such as a document. For example, a user using a computer 102 (FIG. 1)may utilize the table narration program 108A (FIG. 1) to extract thetable contents associated with a table from a source document or webpage so that the extracted table may be narrated.

Then at 8004, the input table may be parsed. As such, the tablestructure and table contents may be parsed to determine whether thetable has certain fields such as a header, columns, or rows, etc.

Next, at 8006 the table structure may be determined and the structuralanalysis of the table may be performed. Therefore, based on the parsingof the table contents from the previous step 804, the table structuremay be analyzed to determine the layout of the table. For example, tablenarration program 108A (FIG. 1) may analyze the previously parsedfields, such as rows or columns that are associated with the table.Furthermore, table narration program 108A (FIG. 1) may determine howmany rows or how many columns are contained in the table. Therefore, thecontents and layout of the table may be determined based on thestructural analysis.

Then at 8008, the structural template that is relevant to or matchingthe given table structure may be determined and selected from the set ofstructural templates (i.e., the structural template repository 722). Assuch, the table structure and the analysis from the previous step 806 iscompared to the structural templates contained in the structuraltemplates repository 722 so that a matching structural templatecontained in the repository 722 may be determined.

Next, at 8010 it is determined whether a structural template is found.The structural template may depict the key elements (i.e., schema) ofthe table structure that may be necessary to determine the applicablenarration templates. If at 8010, it is determined that a structuraltemplate is not found, then the method may terminate. However, if at8010, it is determined that a structural template is found, then at 8012the method may search the “template mapping” file (i.e., templatemapping repository 724) to determine the set of candidate narrationtemplates (from the narration templates repository 726) that may map tothe selected structural template. As such, based on the structuraltemplate determined for the given table, the set of (one or more)applicable narration template candidates may be determined by looking upthe “template mapping” in the template mapping repository 724.Therefore, each structural template may be mapped to one or morenarration templates and each narration template may be mapped to one ormore structural templates.

Then, at 8014 it is determined whether a candidate narration template isfound. If at 8014, it is determined that a candidate narration templateis not found, then the method may terminate. However, if at 8014, it isdetermined that a candidate narration template is found, then at 8016the method may select exactly one narration template if multiplecandidate narration templates were identified. As such, if multiplecandidate narration templates are identified for the selected structuraltemplate, then the method may use the parameters within each narrationtemplate or any additional criteria or rules from the downselectioncriteria/rules repository 8022 to downselect the candidate templatesuntil exactly one narration template is selected. Therefore, for a givenstructural template, if more than one narration template may beapplicable, then additional rules or criteria may be used to downselector select one narration template from the set of applicable narrationtemplate candidates. However, if only one candidate narration templatewas previously identified (from step 8012), then according to oneimplementation, the method may skip step 816 and proceed directly tostep 8018.

Next, at 8018 the method may apply the selected narration template, thestructural template, and the narration formula(s) within the template tothe contents of the given table to determine the “table narration”output. As such, the given table may be narrated by the method of“applying” the selected narration template to the table to get theresultant table narration. Then, at 8020 the method may return the tablenarration output.

Additionally, implementations of the present embodiment may be generic,flexible and scalable, and may be used to narrate a wide variety oftables with different types of structural patterns and content.Therefore, if the implementation or code supports a specific templateschema, then all narration templates based on the defined schema may besupported by the implementation. Additionally, natural languageprocessing (NLP) concepts may be used to define the narration formulas.As such, narration may be improved by updating the narration formula inthe narration template, which may not require any code changes. Also,new narration templates may be defined or existing templates may bemodified without changes to the code.

Furthermore, according to one implementation, schemas may be definedusing existing constructs such as XSD (XML Schema Definition) or DTD(Document Type Descriptor) and templates (structural and narrationtemplates and mapping files) may be defined using XML. Thus, XML and XSDfiles may be treated as “resource files” in a codebase. Additionally,resource files may be updated or new resource files may be added withoutchanges to the existing codebase. As such, the present embodiment mayprovide flexibility for table narration in an ingestion pipeline 718(FIG. 7) for use in a question answering system 720 (FIG. 7), such as,IBM Watson.

Referring now to FIG. 9, examples of narration formulas 9000 inaccordance with one embodiment is depicted. According to at least oneimplementation, narration formulas may be defined at multiple levels.The narration formulas may be implemented as generic, and the narrationformulas may be defined in various ways with different scopes (within atable, within a region comprising multiple tables, etc.). According toat least one implementation of the present embodiment, narrationformulas may be utilized for narrating tables corresponding to specifiedstructural and narration templates. As such, when a structural templateand the applicable narration template are determined for a given table(as previously described), the narration formula specified in theselected narration template can be “applied” to narrate the given table.A narration formula may consist of a mix of natural language words,phrases, or sentences which can be static (specified within the formula)or dynamic (based on the contents of a given table) through NLPtechniques (such as determination of hypernyms or super-subordinaterelationships among cognitive synonyms, preposition classes, plurality,etc.) or based on values specified in other parameters of the narrationtemplate). Furthermore, variables may be resolved to header cells,normal cells, rows, columns or other structural elements of a table,such that the contents of the element specified by the given variablecan be accessed and substituted within the formula. The following areexamples of narration formulas that correlate to the examples depictedin FIG. 9, however, the following examples are generic, and narrationformulas can be defined in various ways with different scopes (within atable, within a region comprising multiple tables, etc.).

-   -   Cell Level Narration Formula 902: This may apply at an        individual cell level, applicable to normal cells and header        cells of any type (normal cells, spanned cells, merged cells,        etc.)    -   Row Level Narration Formula 904: This may apply at an individual        row level, applicable to normal rows. The row-level formula may        include formula elements (notations, words/phases, variables,        etc.) which are based on NLP concepts. For horizontal narration,        the row level formula may be applied to horizontal rows. For        vertical narration, the row level formula may be applied to        columns.    -   Column Level Narration Formula: A separate column-level        narration formula may not be necessary, but may be specified        based on implementation logic. For vertical narration, the row        level narration formula may be applied to columns.    -   Footer Level Narration Formula: This may apply to intermediate        summary rows or footer rows, and is similar to row-level        narration formulas. If the entire footer is a single cell, a        cell-level narration formula can also be applied.

FIG. 10 is a block diagram 800 of internal and external components ofcomputers depicted in FIG. 1 in accordance with an illustrativeembodiment of the present invention. It should be appreciated that FIG.10 provides only an illustration of one implementation and does notimply any limitations with regard to the environments in which differentembodiments may be implemented. Many modifications to the depictedenvironments may be made based on design and implementationrequirements.

Data processing system 800, 900 is representative of any electronicdevice capable of executing machine-readable program instructions. Dataprocessing system 800, 900 may be representative of a smart phone, acomputer system, PDA, or other electronic devices. Examples of computingsystems, environments, and/or configurations that may represented bydata processing system 800, 900 include, but are not limited to,personal computer systems, server computer systems, thin clients, thickclients, hand-held or laptop devices, multiprocessor systems,microprocessor-based systems, network PCs, minicomputer systems, anddistributed cloud computing environments that include any of the abovesystems or devices.

User client computer 102 (FIG. 1), and network server 112 (FIG. 1) mayinclude respective sets of internal components 800 a, b and externalcomponents 900 a,b illustrated in FIG. 10. Each of the sets of internalcomponents 800 a, b includes one or more processors 820, one or morecomputer-readable RAMs 822 and one or more computer-readable ROMs 824 onone or more buses 826, and one or more operating systems 828 and one ormore computer-readable tangible storage devices 830. The one or moreoperating systems 828 and table narration program 108A (FIG. 1) inclient computer 102 and table narration program 108B in network servercomputer 112 are stored on one or more of the respectivecomputer-readable tangible storage devices 830 for execution by one ormore of the respective processors 820 via one or more of the respectiveRAMs 822 (which typically include cache memory). In the embodimentillustrated in FIG. 10, each of the computer-readable tangible storagedevices 830 is a magnetic disk storage device of an internal hard drive.Alternatively, each of the computer-readable tangible storage devices830 is a semiconductor storage device such as ROM 824, EPROM, flashmemory or any other computer-readable tangible storage device that canstore a computer program and digital information.

Each set of internal components 800 a,b, also includes a R/W drive orinterface 832 to read from and write to one or more portablecomputer-readable tangible storage devices 936 such as a CD-ROM, DVD,memory stick, magnetic tape, magnetic disk, optical disk orsemiconductor storage device. A software program, such as tablenarration program 108A and 108B, can be stored on one or more of therespective portable computer-readable tangible storage devices 936, readvia the respective R/W drive or interface 832 and loaded into therespective hard drive 830.

Each set of internal components 800 a, b also includes network adaptersor interfaces 836 such as a TCP/IP adapter cards, wireless Wi-Fiinterface cards, or 3G or 4G wireless interface cards or other wired orwireless communication links. The table narration program 108A in clientcomputer 102 and table narration program 108B in network server 112 canbe downloaded to client computer 102 from an external computer via anetwork (for example, the Internet, a local area network or other, widearea network) and respective network adapters or interfaces 836. Fromthe network adapters or interfaces 836, the table narration program 108Ain client computer 102 and the table narration program 108B in networkserver computer 112 are loaded into the respective hard drive 830. Thenetwork may comprise copper wires, optical fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers.

Each of the sets of external components 900 a,b can include a computerdisplay monitor 920, a keyboard 930, and a computer mouse 934. Externalcomponents 900 a,b can also include touch screens, virtual keyboards,touch pads, pointing devices, and other human interface devices. Each ofthe sets of internal components 800 a,b also includes device drivers 840to interface to computer display monitor 920, keyboard 930 and computermouse 934. The device drivers 840, R/W drive or interface 832 andnetwork adapter or interface 836 comprise hardware and software (storedin storage device 830 and/or ROM 824).

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

1.-8. (canceled)
 9. A computer system for narrating a table using atleast one narration template, wherein the table is extracted from a datasource, the computer system comprising: one or more processors, one ormore computer-readable memories, one or more computer-readable tangiblestorage devices, and program instructions stored on at least one of theone or more storage devices for execution by at least one of the one ormore processors via at least one of the one or more memories, whereinthe computer system is capable of performing a method comprising:parsing the extracted table; performing structural analysis on theparsed extracted table; selecting at least one structural template basedon the structural analysis of the parsed extracted table; selecting theat least one narration template based on the at least one selectedstructural template; applying the at least one selected narrationtemplate to the extracted table; and narrating the extracted table basedon the applying of the at least one selected narration template to theextracted table.
 10. The computer system of claim 9, wherein the atleast one selected structural template comprises at least one keyelement of the table structure that is necessary to determine the atleast one selected narration template.
 11. The computer system of claim9, wherein the at least one selected narration template is based on amapping of the at least one selected structural template to the at leastone selected narration template.
 12. The computer system of claim 9,wherein the at least one selected structural template is based on amapping of the at least one selected narration template to the at leastone selected structural template.
 13. The computer system of claim 9,wherein the at least one selected narration template comprises at leastone key element for defining the at least one selected narrationtemplate.
 14. The computer system of claim 13, wherein the at least onekey element comprises at least one of an natural language processing(NLP) element and a narration formula.
 15. The computer system of claim11, wherein the mapping is based on at least one key element required tomap the least one selected structural template to the at least oneselected narration template.
 16. The computer system of claim 12,wherein the mapping is based on at least one key element required to mapthe at least one selected narration template to the at least oneselected structural template.
 17. A computer program product fornarrating a table using at least one narration template, wherein thetable is extracted from a data source, the computer program productcomprising: one or more computer-readable storage devices and programinstructions stored on at least one of the one or more tangible storagedevices, the program instructions executable by a processor, the programinstructions comprising: program instructions to parse the extractedtable; program instructions to perform structural analysis on the parsedextracted table; program instructions to select at least one structuraltemplate based on the structural analysis of the parsed extracted table;program instructions to select the at least one narration template basedon the at least one selected structural template; program instructionsto apply the at least one selected narration template to the extractedtable; and program instructions to narrate the extracted table based onthe applying of the at least one selected narration template to theextracted table.
 18. The computer program product of claim 17, whereinthe at least one selected structural template comprises at least one keyelement of the table structure that is necessary to determine the atleast one selected narration template.
 19. The computer program productof claim 17, wherein the at least one selected narration template isbased on a mapping of the at least one selected structural template tothe at least one selected narration template.
 20. The computer programproduct of claim 17, wherein the at least one selected structuraltemplate is based on a mapping of the at least one selected narrationtemplate to the at least one selected structural template.